Using multiple criteria decision analysis to aid the selection of enterprise resource planning software : a case study

BHC Ltd is a family owned SME which specialises in steel fabrication for the construction industry. Due to rapid growth over the past decade the company’s current business software has evolved from a collection of semi-integrated individual packages and Excel spreadsheets. To help the company become more efficient during the current financial downturn and to ensure they are capable of future growth, BHC Ltd initiated a project with the University of Strathclyde to select and implement an Enterprise Resource Planning (ERP) solution. This paper will provide a case study of BHC’s ERP selection process. In particular it will discuss how steel specific business requirements and organisational culture led us to use multiple criteria decision analysis (MCDA) when making a final software selection. The MCDA process that was followed is further discussed and includes the success that was achieved by using this approach

Critical learning incidents in system dynamics modelling engagements

This paper reports in-depth behavioural operational research to explore how individual clients learned to resolve dynamically complex problems in system dynamics model-based engagements. Consultant-client dyads were interviewed in ten system dynamics consulting engagements to identify individual clients' Critical Learning Incidents—defined as the moment of surprise caused after one's mental model produces unexpected failure and a change in one’s mental model produces the desired result. The cases are reprised from interviews and include assessments of the nature of the engagement problem, the form of system dynamics model, and the methods employed by consultants during each phase of the engagement. Reported Critical Learning Incidents are noted by engagement phase and consulting method, and constructivist learning theory is used to describe a pattern of learning. Outcomes of the research include describing the role of different methods applied in engagement phases (for example, the role of concept models to commence problem identification and to introduce iconography and jargon to the engagement participants), how model form associates with timings of Critical Learning Incidents, and the role of social mediation and negotiation in the learning process

Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis

Procedural and behavioural biases have received little attention in recent Multi-Criteria Decision Analysis (MCDA) research. Our literature review shows that most research on biases was done 15–30 years ago. This study focuses on biases that are introduced at an early stage of MCDA when building objectives hierarchies and their effect on the weights. The main objective is to investigate whether prior findings regarding such biases, which were mostly based on laboratory experiments, can be found in real-world applications. We conducted a meta-analysis of the objectives hierarchies and weight elicitation procedures in 61 environmental and energy MCDA cases. Relationships between the structural characteristics of the objectives hierarchy and assigned objectives’ weights were analysed with statistical tests. Our main research questions were: (i) How does hierarchy size and structure affect the objectives’ weights? (ii) How are weights distributed across economic, social and environmental objectives? (iii) Is there support for the equalising bias? Our findings are mostly aligned with earlier research and suggest that the hierarchy structure and content can substantially influence weight distributions. For example, hierarchical weighting seems to be sensitive to the asymmetry bias, which can occur when a hierarchy has branches that differ in the number of sub-objectives. We found no evidence for the equalising bias. We highlight issues deserving more attention when developing objectives hierarchies and eliciting weights. The research demonstrates the potential to use meta-analysis, which has not previously been used in this way in the MCDA field, to learn from a collection of applications

Structuring problems for Multi-Criteria Decision Analysis in practice : a literature review of method combinations

Structuring problems for Multi-Criteria Decision Analysis (MCDA) has attracted increasing attention over the past 20 years from both a conceptual and a practical perspective. This is reflected in a significant growth in the number of published applications which use a formal approach to problem structuring in combination with an analytic method for multi-criteria analysis. The problem structuring approaches (PSMs) include general methodologies such as Checkland's Soft Systems Method (SSM), Eden and Ackermann's Strategic Options Design and Analysis (SODA) and other methods that focus on a particular aspect. We carried out a literature review that covers eight PSMs (Cognitive and Causal Maps, DPSIR, Scenario Planning, SSM, Stakeholder Analysis, Strategic Choice Approach, SODA and SWOT) and seven MCDA methods (AHP, ANP, ELECTRE, MAUT, MAVT, PROMETHEE and TOPSIS). We first identified and analysed 333 articles published during 2000-2015, then selected 68 articles covering all PSM-MCDA combinations, which were studied in detail to understand the associated processes, benefits and challenges. The three PSMs most commonly combined with MCDA are SWOT, Scenario Planning and DPSIR. AHP was by far the most commonly applied MCDA method. Combining PSMs with MCDA produces a richer view of the decision situation and enables more effective support for different phases of the decision-making process. Some limitations and challenges in combining PSMs and MCDA are also identified, most importantly relating to building a value tree and assigning criteria weights

Measuring organisational performance using a mix of OR methods

Performance measurement has become an increasingly important issue in recent years. In spite of the remarkable progress that has been achieved in this area of research, many performance measurement initiatives fall short of their potential in supporting decision-making. This paper argues that adopting a multi-method approach to assessing performance has the potential to result in more comprehensive and effective performance measurement systems. To support this assertion, the paper discusses the development of a performance measurement system for a Business Tax Department, which combined the use of several operational research (OR) techniques including qualitative system dynamics, data envelopment analysis and multiple criteria decision analysis. The use of these OR techniques was influential in developing and implementing the performance measurement system and has the potential to be transferred to other contexts

A new collaborative methodology for assessment and management of ecosystem services

© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/)Collaborative management is a new framework to help implement programmes in protected areas. Within this context, the aim of this work is twofold. First, to propose a robust methodology to implement collaborative management focused on ecosystem services. Second, to develop indicators for the main functions of ecosystem services. Decision makers, technical staff and other stakeholders are included in the process from the beginning, by identifying ecosystem services and eliciting preferences using the AHP method. Qualitative and quantitative data are then integrated into a PROMETHEE based method in order to obtain indicators for provisioning, maintenance and direct to citizens services. This methodology, which has been applied in a forest area, provides a tool for exploiting available technical and social data in a continuous process, as well as providing easy to understand graphical results. This approach also overcomes the difficulties found in prioritizing management objectives in a multiple criteria context with limited resources and facilitates consensus between all of the people involved. The new indicators define an innovative approach to assessing the ecosystem services from the supply perspective and provide basic information to help establish payment systems for environmental services and compensation for natural disasters.The authors acknowledge the support received from the Ministry of Economy and Competitiveness through the research project Ref. ECO2011-27369, as well as the time and expert judgments from all the decision makers, technical staff, other stakeholders and citizens involved in the surveys we have carried out. A special mention to the director and technical staff of the Serra d'Espada Natural Park for all support and collaboration with this research. Marina Segura also thanks the Ministry of Education for support through the scholarship of Training Plan of University Teaching for developing her doctoral thesis. We also thank the reviewers for their suggestions to improve the paper.Segura Maroto, M.; Maroto Álvarez, MC.; Belton, V.; Ginestar Peiro, CM. (2015). A new collaborative methodology for assessment and management of ecosystem services. Forests. 6(5):1696-1720. https://doi.org/10.3390/f6051696S1696172065Fitzsimons, J., Pearson, C. J., Lawson, C., & Hill, M. J. (2012). Evaluation of land-use planning in greenbelts based on intrinsic characteristics and stakeholder values. Landscape and Urban Planning, 106(1), 23-34. doi:10.1016/j.landurbplan.2012.01.012Kijazi, M. H., & Kant, S. (2011). Social acceptability of alternative forest regimes in Mount Kilimanjaro, Tanzania, using stakeholder attitudes as metrics of uncertainty. Forest Policy and Economics, 13(4), 242-257. doi:10.1016/j.forpol.2010.12.001Mustajoki, J., Saarikoski, H., Marttunen, M., Ahtikoski, A., Hallikainen, V., Helle, T., … Ylisirniö, A.-L. (2011). Use of decision analysis interviews to support the sustainable use of the forests in Finnish Upper Lapland. Journal of Environmental Management, 92(6), 1550-1563. doi:10.1016/j.jenvman.2011.01.007Palacios-Agundez, I., Casado-Arzuaga, I., Madariaga, I., & Onaindia, M. (2013). The Relevance of Local Participatory Scenario Planning for Ecosystem Management Policies in the Basque Country, Northern Spain. Ecology and Society, 18(3). doi:10.5751/es-05619-180307Tompkins, E. L., Few, R., & Brown, K. (2008). Scenario-based stakeholder engagement: Incorporating stakeholders preferences into coastal planning for climate change. Journal of Environmental Management, 88(4), 1580-1592. doi:10.1016/j.jenvman.2007.07.025Apostolopoulou, E., Drakou, E. G., & Pediaditi, K. (2012). Participation in the management of Greek Natura 2000 sites: Evidence from a cross-level analysis. Journal of Environmental Management, 113, 308-318. doi:10.1016/j.jenvman.2012.09.006Liu, J., Ouyang, Z., & Miao, H. (2010). Environmental attitudes of stakeholders and their perceptions regarding protected area-community conflicts: A case study in China. Journal of Environmental Management, 91(11), 2254-2262. doi:10.1016/j.jenvman.2010.06.007Rees, S. E., Attrill, M. J., Austen, M. C., Mangi, S. C., & Rodwell, L. D. (2013). A thematic cost-benefit analysis of a marine protected area. Journal of Environmental Management, 114, 476-485. doi:10.1016/j.jenvman.2012.10.048Reed, M. S. (2008). Stakeholder participation for environmental management: A literature review. Biological Conservation, 141(10), 2417-2431. doi:10.1016/j.biocon.2008.07.014Mendoza, G. A., & Martins, H. (2006). Multi-criteria decision analysis in natural resource management: A critical review of methods and new modelling paradigms. Forest Ecology and Management, 230(1-3), 1-22. doi:10.1016/j.foreco.2006.03.023De Brucker, K., Macharis, C., & Verbeke, A. (2013). Multi-criteria analysis and the resolution of sustainable development dilemmas: A stakeholder management approach. European Journal of Operational Research, 224(1), 122-131. doi:10.1016/j.ejor.2012.02.021Petrokofsky, G., Brown, N. D., Hemery, G. E., Woodward, S., Wilson, E., Weatherall, A., … Arnold, J. E. M. (2010). A participatory process for identifying and prioritizing policy-relevant research questions in natural resource management: a case study from the UK forestry sector. Forestry, 83(4), 357-367. doi:10.1093/forestry/cpq018Maroto Álvarez, C., Segura, M., Ginestar, C., Uriol, J., & Segura, B. (2013). Sustainable Forest Management in a Mediterranean region: Social preferences. Forest Systems, 22(3), 546. doi:10.5424/fs/2013223-04135Segura, M., Ray, D., & Maroto, C. (2014). Decision support systems for forest management: A comparative analysis and assessment. Computers and Electronics in Agriculture, 101, 55-67. doi:10.1016/j.compag.2013.12.005Martinez-Harms, M. J., Bryan, B. A., Balvanera, P., Law, E. A., Rhodes, J. R., Possingham, H. P., & Wilson, K. A. (2015). Making decisions for managing ecosystem services. Biological Conservation, 184, 229-238. doi:10.1016/j.biocon.2015.01.024Sijtsma, F. J., van der Heide, C. M., & van Hinsberg, A. (2013). Beyond monetary measurement: How to evaluate projects and policies using the ecosystem services framework. Environmental Science & Policy, 32, 14-25. doi:10.1016/j.envsci.2012.06.016Macharis, C., Springael, J., De Brucker, K., & Verbeke, A. (2004). PROMETHEE and AHP: The design of operational synergies in multicriteria analysis. European Journal of Operational Research, 153(2), 307-317. doi:10.1016/s0377-2217(03)00153-xFontana, V., Radtke, A., Bossi Fedrigotti, V., Tappeiner, U., Tasser, E., Zerbe, S., & Buchholz, T. (2013). Comparing land-use alternatives: Using the ecosystem services concept to define a multi-criteria decision analysis. Ecological Economics, 93, 128-136. doi:10.1016/j.ecolecon.2013.05.007Turcksin, L., Bernardini, A., & Macharis, C. (2011). A combined AHP-PROMETHEE approach for selecting the most appropriate policy scenario to stimulate a clean vehicle fleet. Procedia - Social and Behavioral Sciences, 20, 954-965. doi:10.1016/j.sbspro.2011.08.104Macharis, C., Turcksin, L., & Lebeau, K. (2012). Multi actor multi criteria analysis (MAMCA) as a tool to support sustainable decisions: State of use. Decision Support Systems, 54(1), 610-620. doi:10.1016/j.dss.2012.08.008Behzadian, M., Kazemzadeh, R. B., Albadvi, A., & Aghdasi, M. (2010). PROMETHEE: A comprehensive literature review on methodologies and applications. European Journal of Operational Research, 200(1), 198-215. doi:10.1016/j.ejor.2009.01.021Huang, I. B., Keisler, J., & Linkov, I. (2011). Multi-criteria decision analysis in environmental sciences: Ten years of applications and trends. Science of The Total Environment, 409(19), 3578-3594. doi:10.1016/j.scitotenv.2011.06.022Ananda, J., & Herath, G. (2009). A critical review of multi-criteria decision making methods with special reference to forest management and planning. Ecological Economics, 68(10), 2535-2548. doi:10.1016/j.ecolecon.2009.05.010Vacik, H., Kurttila, M., Hujala, T., Khadka, C., Haara, A., Pykäläinen, J., … Tikkanen, J. (2014). Evaluating collaborative planning methods supporting programme-based planning in natural resource management. Journal of Environmental Management, 144, 304-315. doi:10.1016/j.jenvman.2014.05.029Fisher, B., & Kerry Turner, R. (2008). Ecosystem services: Classification for valuation. Biological Conservation, 141(5), 1167-1169. doi:10.1016/j.biocon.2008.02.019Corner, J., Buchanan, J., & Henig, M. (2001). Dynamic decision problem structuring. Journal of Multi-Criteria Decision Analysis, 10(3), 129-141. doi:10.1002/mcda.295Marques-Perez, I., Segura, B., & Maroto, C. (2014). Evaluating the functionality of agricultural systems: social preferences for multifunctional peri-urban agriculture. The «Huerta de Valencia» as case study. Spanish Journal of Agricultural Research, 12(4), 889. doi:10.5424/sjar/2014124-6061Xu, Z. (2000). On consistency of the weighted geometric mean complex judgement matrix in AHP. European Journal of Operational Research, 126(3), 683-687. doi:10.1016/s0377-2217(99)00082-xBanville, C., Landry, M., Martel, J.-M., & Boulaire, C. (1998). A stakeholder approach to MCDA. Systems Research and Behavioral Science, 15(1), 15-32. doi:10.1002/(sici)1099-1743(199801/02)15:13.0.co;2-bHarrison, S. R., & Qureshi, M. E. (2000). Choice of stakeholder groups and members in multicriteria decision models. Natural Resources Forum, 24(1), 11-19. doi:10.1111/j.1477-8947.2000.tb00925.xScholten, L., Scheidegger, A., Reichert, P., Mauer, M., & Lienert, J. (2014). Strategic rehabilitation planning of piped water networks using multi-criteria decision analysis. Water Research, 49, 124-143. doi:10.1016/j.watres.2013.11.017Miller, K. A., & Belton, V. (2014). Water resource management and climate change adaptation: a holistic and multiple criteria perspective. Mitigation and Adaptation Strategies for Global Change, 19(3), 289-308. doi:10.1007/s11027-013-9537-0The OECD Glossary of Statistical Terms http://stats.oecd.org/glossary/Parcs Naturals de la Comunitat Valenciana http://www.cma.gva.es/webdoc/documento.ashx?id=141553The List of Wetlands of International Importance http://www.ramsar.orgDe Groot, R. S., Wilson, M. A., & Boumans, R. M. . (2002). A typology for the classification, description and valuation of ecosystem functions, goods and services. Ecological Economics, 41(3), 393-408. doi:10.1016/s0921-8009(02)00089-7Costanza, R., d’ Arge, R., de Groot, R., Farber, S., Grasso, M., Hannon, B., … van den Belt, M. (1998). The value of the world’s ecosystem services and natural capital. Ecological Economics, 25(1), 3-15. doi:10.1016/s0921-8009(98)00020-2Plan de Acción Territorial Forestal de la Comunitat Valenciana http://www.cma.gva.es/webdoc/documento.ashx?id=144302Tofallis, C. (2014). Add or Multiply? A Tutorial on Ranking and Choosing with Multiple Criteria. INFORMS Transactions on Education, 14(3), 109-119. doi:10.1287/ited.2013.0124Valls-Donderis, P., Vallés, M. C., & Galiana, F. (2015). Criteria and indicators for sustainable forestry under Mediterranean conditions applicable in Spain at the forest management unit scale. Forest Systems, 24(1), 004. doi:10.5424/fs/2015241-05542Ananda, J. (2007). Implementing Participatory Decision Making in Forest Planning. Environmental Management, 39(4), 534-544. doi:10.1007/s00267-006-0031-2Mendelsohn, R., & Olmstead, S. (2009). The Economic Valuation of Environmental Amenities and Disamenities: Methods and Applications. Annual Review of Environment and Resources, 34(1), 325-347. doi:10.1146/annurev-environ-011509-135201Wilson, M. A., & Hoehn, J. P. (2006). Valuing environmental goods and services using benefit transfer: The state-of-the art and science. Ecological Economics, 60(2), 335-342. doi:10.1016/j.ecolecon.2006.08.01

Methods to inform the development of concise objectives hierarchies in multi-criteria decision analysis

Building a well-structured objectives hierarchy is central to multi-criteria decision analysis (MCDA). However, in the absence of a systematic methodology to support the process, this task has been described as “more art than science”. Objectives hierarchies often tend to become large and constraining the size of a hierarchy can be challenging. This paper proposes and illustrates the use of a set of methods to support the simplification of the hierarchies in contexts that are “datarich” and characterised by many objectives. The aim of using the proposed approach is to support decision analysts in developing an appropriately concise decision model for the further interactions with the stakeholders. Using data from two completed environmental cases we show retrospectively how qualitative (means-ends networks), semiquantitative (relevancyanalysis) and quantitative (correlation analysis, principal component analysis, local sensitivity analysis of weights) methods, used alone or in combination, can inform hierarchy development. We evaluate the potential benefits and challenges of each method and discuss the advantages and disadvantages of the simplification of an objectives hierarchy. Questionnaire-based relevancy analysis can be a useful method to identify and communicate important objectives in the early phases of an MCDA process with stakeholders, while correlation analysis can help to identify overlapping objectives, particularly in cases having many objectives and alternatives. It is intended that the methods support a facilitator in developing a clear understanding of the problem and also prompt deeper thinking about and discussion of the appropriate structure and content of an objectives hierarchy with the stakeholders involved